Example Notebook for Correcting and loading FastCCD Images¶
This example reads data from the FastCCD and corrects and rotates the images. Included are some example of widgets to interact with the data
Getting Started¶
Load the databroker moudle, csxtools and various other
dependencies
import numpy as np
from databroker import DataBroker, get_table, get_images
from csxtools.utils import get_fastccd_images
from csxtools.ipynb import image_stack_to_movie, show_image_stack
%matplotlib inline
from matplotlib import pyplot as plt
Set the logger level to info to see logging output
import logging
logging.getLogger().setLevel(logging.INFO)
Get the data!¶
Load the headers without the images (using fill=False)
bgnd = DataBroker[57512] #[52440]
print(bgnd)
get_table(bgnd, fill=False)
INFO:databroker.databroker:Interpreting key = 57512 as an integer
header
======
EventDescriptor
---------------
configuration :
ring_curr : value
ring_curr : timestamp
dtype : number
source : PV:XF:23ID-SR{}I-I
upper_ctrl_limit: 0.0
lower_ctrl_limit: 0.0
units : mA
shape : []
precision : 2
sclr_preset_time: value
sclr_preset_time: timestamp
dtype : number
source : PV:XF:23ID1-ES{Sclr:1}.TP
upper_ctrl_limit: 0.0
lower_ctrl_limit: 0.0
units :
shape : []
precision : 3
+--------------------------+---------+------------------+------------+---------------------+-------------+-----------+-----------------+--------------------------------------------+-------+------------------+
| data keys | dtype | enum_strs | external | lower_ctrl_limit | object_name | precision | shape | source | units | upper_ctrl_limit |
+--------------------------+---------+------------------+------------+---------------------+-------------+-----------+-----------------+--------------------------------------------+-------+------------------+
| fccd_acquire_time | number | | | 0.0 | fccd | 3 | [] | PV:XF:23ID1-ES{FCCD}cam1:AcquireTime_RBV | | 0.0 |
| fccd_image | array | | FILESTORE: | | fccd | | [1152, 2050, 0] | PV:XF:23ID1-ES{FCCD} | | |
| fccd_num_images_captured | integer | | | | fccd | | [] | PV:XF:23ID1-ES{FCCD}HDF1:NumCaptured_RBV | | |
| fccd_plugin_num_images | number | | | 0.0 | fccd | 0 | [] | PV:XF:23ID1-ES{FCCD}FastCCD1:NumImages_RBV | | 0.0 |
| fccd_stats1_total | number | | | | fccd | 0 | [] | PV:XF:23ID1-ES{FCCD}Stats1:Total_RBV | | |
| fccd_stats2_total | number | | | | fccd | 0 | [] | PV:XF:23ID1-ES{FCCD}Stats2:Total_RBV | | |
| fccd_stats3_total | number | | | | fccd | 0 | [] | PV:XF:23ID1-ES{FCCD}Stats3:Total_RBV | | |
| fccd_stats4_total | number | | | | fccd | 0 | [] | PV:XF:23ID1-ES{FCCD}Stats4:Total_RBV | | |
| fccd_stats5_total | number | | | | fccd | 0 | [] | PV:XF:23ID1-ES{FCCD}Stats5:Total_RBV | | |
| m1a_pit_actuate | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC | | 0 |
| m1a_pit_done | integer | ['Done', 'Busy'] | | | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}BUSY_STS | | |
| m1a_pit_readback | number | | | | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Pit}Mtr_MON | mrad | |
| m1a_pit_setpoint | number | | | -20.0 | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Pit}Mtr_POS_SP | mrad | 20.0 |
| m1a_pit_stop_signal | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC | | 0 |
| m1a_rol_actuate | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC | | 0 |
| m1a_rol_done | integer | ['Done', 'Busy'] | | | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}BUSY_STS | | |
| m1a_rol_readback | number | | | | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Rol}Mtr_MON | mrad | |
| m1a_rol_setpoint | number | | | -20.0 | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Rol}Mtr_POS_SP | mrad | 20.0 |
| m1a_rol_stop_signal | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC | | 0 |
| m1a_x_actuate | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC | | 0 |
| m1a_x_done | integer | ['Done', 'Busy'] | | | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}BUSY_STS | | |
| m1a_x_readback | number | | | | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:X}Mtr_MON | mm | |
| m1a_x_setpoint | number | | | -15.0 | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:X}Mtr_POS_SP | mm | 15.0 |
| m1a_x_stop_signal | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC | | 0 |
| m1a_y_actuate | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC | | 0 |
| m1a_y_done | integer | ['Done', 'Busy'] | | | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}BUSY_STS | | |
| m1a_y_readback | number | | | | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Y}Mtr_MON | mm | |
| m1a_y_setpoint | number | | | -10.0 | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Y}Mtr_POS_SP | mm | 10.0 |
| m1a_y_stop_signal | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC | | 0 |
| m1a_yaw_actuate | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC | | 0 |
| m1a_yaw_done | integer | ['Done', 'Busy'] | | | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}BUSY_STS | | |
| m1a_yaw_readback | number | | | | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Yaw}Mtr_MON | mrad | |
| m1a_yaw_setpoint | number | | | -20.0 | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Yaw}Mtr_POS_SP | mrad | 20.0 |
| m1a_yaw_stop_signal | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC | | 0 |
| m1a_z_actuate | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC | | 0 |
| m1a_z_done | integer | ['Done', 'Busy'] | | | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}BUSY_STS | | |
| m1a_z_readback | number | | | | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Z}Mtr_MON | mm | |
| m1a_z_setpoint | number | | | -30.0 | m1a | 3 | [] | PV:XF:23IDA-OP:1{Mir:1-Ax:Z}Mtr_POS_SP | mm | 30.0 |
| m1a_z_stop_signal | integer | | | 0 | m1a | | [] | PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC | | 0 |
| m3a_bdr_user_readback | number | | | | m3a | 4 | [] | PV:XF:23ID1-OP{Mir:3-Ax:Bdr}Mtr.RBV | mm | |
| m3a_bdr_user_setpoint | number | | | -10.0 | m3a | 4 | [] | PV:XF:23ID1-OP{Mir:3-Ax:Bdr}Mtr.VAL | mm | 1.0 |
| m3a_pit_user_readback | number | | | | m3a | 4 | [] | PV:XF:23ID1-OP{Mir:3-Ax:P}Mtr.RBV | mrad | |
| m3a_pit_user_setpoint | number | | | 0.0 | m3a | 4 | [] | PV:XF:23ID1-OP{Mir:3-Ax:P}Mtr.VAL | mrad | 0.0 |
| m3a_x_user_readback | number | | | | m3a | 4 | [] | PV:XF:23ID1-OP{Mir:3-Ax:XAvg}Mtr.RBV | mm | |
| m3a_x_user_setpoint | number | | | -10.0 | m3a | 4 | [] | PV:XF:23ID1-OP{Mir:3-Ax:XAvg}Mtr.VAL | mm | 10.0 |
| ring_curr | number | | | 0.0 | ring_curr | 2 | [] | PV:XF:23ID-SR{}I-I | mA | 0.0 |
| sclr_ch1 | number | | | | sclr | 0 | [] | PV:XF:23ID1-ES{Sclr:1}.S1 | | |
| sclr_ch2 | number | | | | sclr | 0 | [] | PV:XF:23ID1-ES{Sclr:1}.S2 | | |
| sclr_ch3 | number | | | | sclr | 0 | [] | PV:XF:23ID1-ES{Sclr:1}.S3 | | |
| sclr_ch4 | number | | | | sclr | 0 | [] | PV:XF:23ID1-ES{Sclr:1}.S4 | | |
| sclr_ch5 | number | | | | sclr | 0 | [] | PV:XF:23ID1-ES{Sclr:1}.S5 | | |
| sclr_ch6 | number | | | | sclr | 0 | [] | PV:XF:23ID1-ES{Sclr:1}.S6 | | |
| sclr_time | number | | | 0.0 | sclr | 3 | [] | PV:XF:23ID1-ES{Sclr:1}.T | | 0.0 |
| slt1_xc_user_readback | number | | | | slt1 | 4 | [] | PV:XF:23ID1-OP{Slt:1-Ax:XCtr}Mtr.RBV | mm | |
| slt1_xc_user_setpoint | number | | | -10.0 | slt1 | 4 | [] | PV:XF:23ID1-OP{Slt:1-Ax:XCtr}Mtr.VAL | mm | 10.0 |
| slt1_xg_user_readback | number | | | | slt1 | 4 | [] | PV:XF:23ID1-OP{Slt:1-Ax:XGap}Mtr.RBV | mm | |
| slt1_xg_user_setpoint | number | | | -30.000000000000004 | slt1 | 4 | [] | PV:XF:23ID1-OP{Slt:1-Ax:XGap}Mtr.VAL | mm | 30.0 |
| slt1_yc_user_readback | number | | | | slt1 | 4 | [] | PV:XF:23ID1-OP{Slt:1-Ax:YCtr}Mtr.RBV | mm | |
| slt1_yc_user_setpoint | number | | | 1.4003 | slt1 | 4 | [] | PV:XF:23ID1-OP{Slt:1-Ax:YCtr}Mtr.VAL | mm | 1.4003 |
| slt1_yg_user_readback | number | | | | slt1 | 4 | [] | PV:XF:23ID1-OP{Slt:1-Ax:YGap}Mtr.RBV | mm | |
| slt1_yg_user_setpoint | number | | | -20.000899999999998 | slt1 | 4 | [] | PV:XF:23ID1-OP{Slt:1-Ax:YGap}Mtr.VAL | mm | 19.9991 |
| slt2_xc_user_readback | number | | | | slt2 | 4 | [] | PV:XF:23ID1-OP{Slt:2-Ax:XCtr}Mtr.RBV | mm | |
| slt2_xc_user_setpoint | number | | | -0.069 | slt2 | 4 | [] | PV:XF:23ID1-OP{Slt:2-Ax:XCtr}Mtr.VAL | mm | -0.069 |
| slt2_xg_user_readback | number | | | | slt2 | 4 | [] | PV:XF:23ID1-OP{Slt:2-Ax:XGap}Mtr.RBV | mm | |
| slt2_xg_user_setpoint | number | | | 14.2938 | slt2 | 4 | [] | PV:XF:23ID1-OP{Slt:2-Ax:XGap}Mtr.VAL | mm | 14.2938 |
| slt2_yc_user_readback | number | | | | slt2 | 4 | [] | PV:XF:23ID1-OP{Slt:2-Ax:YCtr}Mtr.RBV | mm | |
| slt2_yc_user_setpoint | number | | | 0.1515 | slt2 | 4 | [] | PV:XF:23ID1-OP{Slt:2-Ax:YCtr}Mtr.VAL | mm | 0.1515 |
| slt2_yg_user_readback | number | | | | slt2 | 4 | [] | PV:XF:23ID1-OP{Slt:2-Ax:YGap}Mtr.RBV | mm | |
| slt2_yg_user_setpoint | number | | | 14.6062 | slt2 | 4 | [] | PV:XF:23ID1-OP{Slt:2-Ax:YGap}Mtr.VAL | mm | 14.6062 |
| slt3_x_user_readback | number | | | | slt3 | 4 | [] | PV:XF:23ID1-OP{Slt:3-Ax:X}Mtr.RBV | mm | |
| slt3_x_user_setpoint | number | | | -1000.0 | slt3 | 4 | [] | PV:XF:23ID1-OP{Slt:3-Ax:X}Mtr.VAL | mm | 1000.0 |
| slt3_y_user_readback | number | | | | slt3 | 4 | [] | PV:XF:23ID1-OP{Slt:3-Ax:Y}Mtr.RBV | mm | |
| slt3_y_user_setpoint | number | | | 0.0 | slt3 | 4 | [] | PV:XF:23ID1-OP{Slt:3-Ax:Y}Mtr.VAL | mm | 0.0 |
| temp_a | number | | | | temp | 3 | [] | PV:XF:23ID1-ES{TCtrl:1-Chan:A}T-I | K | |
| temp_b | number | | | | temp | 3 | [] | PV:XF:23ID1-ES{TCtrl:1-Chan:B}T-I | K | |
+--------------------------+---------+------------------+------------+---------------------+-------------+-----------+-----------------+--------------------------------------------+-------+------------------+
run_start : b636d0b5-28be-4280-a7e7-01e76fdb30f8
time : 1455378251.731183
uid : aa53ecdf-a1c4-40a1-9fe6-bf0aac32a692
RunStart
--------
beamline_id : CSX-1
config :
detectors : ['temp', 'slt1', 'slt2', 'slt3', 'm1a', 'm3a', 'ring_curr', 'sclr', 'fccd']
group :
owner : xf23id1
plan_args :
detectors : [Temperature(prefix='XF:23ID1-ES{TCtrl:1', name='temp', read_attrs=['a', 'b'], configuration_attrs=[], monitor_attrs=[]), SlitsGapCenter(prefix='XF:23ID1-OP{Slt:1', name='slt1', read_attrs=['xg', 'xc', 'yg', 'yc'], configuration_attrs=[], monitor_attrs=[]), SlitsGapCenter(prefix='XF:23ID1-OP{Slt:2', name='slt2', read_attrs=['xg', 'xc', 'yg', 'yc'], configuration_attrs=[], monitor_attrs=[]), SlitsXY(prefix='XF:23ID1-OP{Slt:3', name='slt3', read_attrs=['x', 'y'], configuration_attrs=[], monitor_attrs=[]), Mirror(prefix='XF:23IDA-OP:1{Mir:1', name='m1a', read_attrs=['z', 'y', 'x', 'pit', 'yaw', 'rol'], configuration_attrs=[], monitor_attrs=[]), MotorMirror(prefix='XF:23ID1-OP{Mir:3', name='m3a', read_attrs=['x', 'pit', 'bdr'], configuration_attrs=[], monitor_attrs=[]), EpicsSignal(read_pv='XF:23ID-SR{}I-I', name='ring_curr', value=175.2972070250072, timestamp=1455378190.755994, pv_kw={}, auto_monitor=None, string=False, write_pv='XF:23ID-SR{}I-I', limits=False, put_complete=False), EpicsScaler(prefix='XF:23ID1-ES{Sclr:1}', name='sclr', read_attrs=['channels', 'time'], configuration_attrs=['preset_time', 'presets', 'gates'], monitor_attrs=[]), ProductionCam(prefix='XF:23ID1-ES{FCCD}', name='fccd', read_attrs=['hdf5', 'acquire_time', 'num_images_captured', 'plugin_num_images', 'stats1', 'stats2', 'stats3', 'stats4', 'stats5'], configuration_attrs=[], monitor_attrs=[])]
delay : 0
num : 1
plan_type : Count
project : Prop. 300121, SAF = 300275
sample :
type : small
composition : LBCO 1/8
scan_id : 57512
time : 1455378191.0151987
uid : b636d0b5-28be-4280-a7e7-01e76fdb30f8
RunStop
-------
exit_status : success
reason :
run_start : b636d0b5-28be-4280-a7e7-01e76fdb30f8
time : 1455378251.826415
uid : 01ad412f-59e5-4ff1-84bf-dca265ca3f77
| time | m3a_x_user_setpoint | slt1_xg_user_setpoint | slt3_y_user_readback | slt1_yg_user_readback | slt2_yg_user_setpoint | m3a_pit_user_setpoint | slt3_x_user_readback | m1a_yaw_readback | m1a_y_actuate | ... | m1a_yaw_setpoint | slt2_yc_user_readback | m1a_y_readback | slt2_xg_user_readback | slt2_yg_user_readback | fccd_stats1_total | m1a_z_setpoint | m1a_rol_done | sclr_time | m1a_x_readback | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2016-02-13 15:44:11.782440-05:00 | 0 | 5 | -0.25073 | 5.0001 | -1.776357e-15 | -0.616666 | -6.0802 | 0.000091 | 1 | ... | 0 | -5.0001 | -2.409994 | 10 | -1.776357e-15 | 1451 | -27.62 | 0 | 1 | 0.000024 |
1 rows × 76 columns
data = DataBroker[57511] #[52436]
get_table(data, fill=False)
INFO:databroker.databroker:Interpreting key = 57511 as an integer
| time | m3a_x_user_setpoint | slt1_xg_user_setpoint | slt3_y_user_readback | slt1_yg_user_readback | slt2_yg_user_setpoint | m3a_pit_user_setpoint | slt3_x_user_readback | m1a_yaw_readback | m1a_y_actuate | ... | m1a_yaw_setpoint | slt2_yc_user_readback | m1a_y_readback | slt2_xg_user_readback | slt2_yg_user_readback | fccd_stats1_total | m1a_z_setpoint | m1a_rol_done | sclr_time | m1a_x_readback | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2016-02-13 15:40:56.780111-05:00 | 0 | 5 | -0.25073 | 5.0002 | -1.776357e-15 | -0.616666 | -6.08021 | 0.000091 | 1 | ... | 0 | -5.0001 | -2.409994 | 10 | -1.776357e-15 | 1360683 | -27.62 | 0 | 1 | 0.000024 |
1 rows × 76 columns
Correct the FastCCD Images¶
Now correct the CCD images, using only a single background image.
In the future this should be taken from the metadata associated with the data, or from the darkfield entry
First lets just look at the background stack as a “lightfield”
images = get_fastccd_images(data, (bgnd, None, None), tag='fccd_image')
INFO:csxtools.utils:Took 1.64s to read data using get_images
INFO:csxtools.fastccd.images:Correcting image stack of shape (600, 960, 960)
INFO:csxtools.fastccd.images:Not correcting for darkfield. No input.
INFO:csxtools.fastccd.images:Not correcting for flatfield. No input.
INFO:csxtools.fastccd.images:Corrected image stack in 0.359s
WARNING:csxtools.utils:Missing dark image for gain setting 1
WARNING:csxtools.utils:Missing dark image for gain setting 2
INFO:csxtools.utils:Computed dark images in 17.6s
INFO:csxtools.utils:Took 28.7s to read data using get_images
INFO:csxtools.fastccd.images:Correcting image stack of shape (6000, 960, 960)
INFO:csxtools.fastccd.images:Not correcting for flatfield. No input.
(1, 600, 960, 960) (3, 960, 960) (960, 960)
(1, 6000, 960, 960)
INFO:csxtools.fastccd.images:Corrected image stack in 3.31s
(3, 960, 960) (960, 960)
images.shape
(1, 6000, 960, 960)
Now lets do an interactive plotting widget!¶
Thanks ipywidgets!
show_image_stack(np.log10(images[0]), (-10, 10000))
And lets do a movie of the stack of images¶
This produces an HTML5 movie which can be viewed in most browsers
image_stack_to_movie(images[0], frames=100, figsize=(12, 10), vmin=-20, vmax=300)
What version are we running?
import csxtools
csxtools.__version__
images.shape
(1, 6000, 960, 960)
plt.imshow(np.log10(np.nanmean(images[0][0:100,400:480,520:630], axis=0)), vmin=-3, vmax=3)
<matplotlib.image.AxesImage at 0x7fb3ed7121d0>
stack =
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-30-a30db1dae88d> in <module>()
----> 1 stack = np.log10(np.nanmean(images[0][0:100,400:480,520:630], axis=0), vmin=0, vmax=1)
TypeError: 'vmax' is an invalid keyword to ufunc 'log10'
np.nanmax(stack)
3.9225066
800/(2.8*6)
47.61904761904763